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Abstract 



We propose a perturbation method for determining the (largest) group of invari- 
ance of a toric ideal defined in Q]- In the perturbation method, we investigate how a 
generic element in the row space of the configuration defining a toric ideal is mapped 
by a permutation of the indeterminates. Compared to the proof in 0] which was 
based on stabilizers of a subset of indeterminates, the perturbation method gives 
a much simpler proof of the group of invariance. In particular, we determine the 
group of invariance for a general hierarchical model of contingency tables in statis- 
tics, under the assumption that the numbers of the levels of the factors are generic. 
We prove that it is a wreath product indexed by a poset related to the intersection 
poset of the maximal interaction effects of the model. 

Key words and phrases: computational algebraic statistics, group action, stabilizer, su- 
doku, wreath product. 

Math Subject Classification (2000): 62H17, 05E20. 



2 



1 Introduction 



Since the introduction of the notion of Markov basis by [5(], toric ideals associated with 
various statistical models have been intensively investigated by both statisticians and al- 
gebraists. In particular, statistical models for contingency tables have been rich sources 



for new developments (e.g. 11], 6|], 13|]). The most important statistical model for contin- 
gency tables is the hierarchical model (e.g. Ill]), which describes interactions of factors 
in terms an abstract simplicial complex. The configuration and the toric ideal associated 
with a hierarchical model is highly symmetric. Therefore it is of considerable interest to 
determine the (largest) group of invariance of a general hierarchical model. The group 
of invariance is the set of permutations of the cells of contingency tables (or the inde- 
terminates of a polynomial ring) which leaves the model (or, equivalently, the kernel of 
the configuration, or the row space of the configuration) invariant. Once the group of 



invariance is determined, a Markov basis (or equivalent 



y a system of binomial generators 



by a list of representative 



of the toric ideal) can be very concisely described 
elements from the orbits of the group. Without the consideration of symmetry, Markov 
bases for statistical problems tend to be very large (e.g. 8|]). 

Given a particular statistical model it is often easy to guess a candidate group, under 
which the model is clearly invariant. However as shown in [2] it is often difficult to prove 
that it is the largest group of invariance, i.e., every permutation outside the group does not 
leave the model invariant. In this paper we propose a perturbation method to determine 
the group of invariance. In this method, we look at a generic element of the model and 
check if a permutation maps the element to another element in the model. The candidate 
group is shown to be the largest group of invariance, if every permutation which maps 
a sufficiently generic element of the model into the model is necessarily an element of 
the candidate group. In order to show the effectiveness of this approach, we determine 
the group of invariance for a general hierarchical model of contingency tables, under the 



assumption that the numbers of the levels of the factors are generic. We prove that the 
group of invariance is a wreath product indexed by a poset related to the intersection 
poset of the maximal interaction effects of the hierarchical model. 

Here we give a simple illustrative example. Consider a hierarchical model for four- 
factor contingency tables with numbers of levels Ji, I2, I3 and I4 and the set of facets 
{{1, 2}, {2, 3}, {3, 4}} (see Section [2] for details of notation and terminology). Our main 
theorems (Theorem [1] and Theorem [21) state that under a weak regularity condition the 
group of invariance of this model is generated by the permutations of i2, the permutations 
of 23, the permutations of ii depending on 22 and the permutations of 24 depending on 
23. This group is strictly larger than the direct-product group Sj^ X X (S/g X Sj^ 
of permutations of levels for each factor. Other examples are given in Section [61 In 
particular, we present an example such that the number of orbits in the minimal Markov 
basis is smaller under the action of the group of invariance than under the action of the 
direct-product group (see Example [6]). 

In our proof we need to establish some basic facts on hierarchical models, which are 
not found in the existing statistical literature. These facts are of independent interest and 
we present them in Section [H 

The organization of the paper is as follows. In Section [21 we give preliminaries and 
present a perturbation lemma. In Section [31 we state our main theorem, which expresses 
the group of invariance of a hierarchical model as an intersection of wreath products of 
symmetric groups. In Section [H we establish some basic facts on hierarchical models and 
in Section [51 we give a proof of the main theorem. In Section [61 we rewrite the group of 
invariance as a wreath product indexed by a poset related to the intersection poset of the 
maximal interaction effects of the hierarchical model. We conclude the paper by some 
discussions in Section [71 
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2 Preliminaries and a perturbation lemma 

In this section we summarize preliminary facts on hierarchical models for contingency 
tables, define the group of invariance and present a perturbation lemma, which is essential 
for our proofs. We mainly follow the notation and terminology of 11 1. 



2.1 Preliminaries on hierarchical models for contingency tables 

A hierarchical model for m-factor contingency tables with numbers of levels Ji, . . . ,7^ 
is specified by an abstract simplicial complex. Let A be an abstract simplicial complex 



id . Section 2.1]) of subsets of a finite set {1, . . . , m} = [m] of "factors". We denote the 
set of maximal simplices of A by facet (A) = {Di, . . . , Dk}- Maximal simplices are called 
maximal interaction effects of the model. For each factor j G [m], the set of "levels" of 
j is denoted by X,- = {1, . . . , 1^} = [Ij], where Ij > 2. The direct product of the set of 
levels X = Xi X ■ ■ ■ X X^ is the set of "cells" and its element i = {ii, . . . , im) is a cell. A 
contingency table x = {x{i))i^x is a vector of nonnegative integers indexed by the cells. 
The number x{i) is the frequency of the cell *. In this paper, the symbol A G B means 
that y4 is a subset of B. If A is a proper subset of B, then we write A C. B. 

For a subset D C [m] of factors, let Jo = Y[j(z£,Ij- A subvector of indices Id = 
{ij)j^D G Td is called "a marginal cell". When a particular cell i = (ii, . . . ,im) is given, 
io is regarded as a subvector of i, i.e., the projection of i onto the coordinates in D. For 
a contingency table x, its D-marginal table x~^ = {x~^{ir)))ioeio is defined by 

JGI, 3D=iD 

Similar notation is used even when x{i) is not necessarily a nonnegative integer. 

Fix X and a hierarchical model A with facet (A) = {-Di, . . . , D^}- Write v =_y2!k=i \-^d^, I 



and p = |X|. For each i = {ii, . . . , im) G X consider the following vector (cf. 131 ]) 



where e^''\i£i^) is a unit coordinate vector of dimension |Xd^, | with 1 at the position i^^. 
and everywhere else. The configuration for A is the set of p vectors 



AA = {eW(i^J©---©e(^)(iz),-)} 



In this paper we regard as a z/ x p integral matrix representing a linear map from Q-^ 
to Q*^. The matrix A^ can also be expressed by Kronecker products of identity matrices 
and vectors consisting of I's ([l^, Section 2.1]). We also assume that the domain is 
equipped with the standard inner product and we identify with its dual space by the 
standard inner product. 



Let {ui}i(zx be the set of indeterminates indexed by the cells and let {tf^ }io-^eiDi 



U- ■ -U 



}*dj^.gXi3^ denote the set of indeterminates indexed by the rows of Aa- The toric ideal 
/aa is the kernel of the polynomial homomorphism tta defined by iTA{ui) 
The structure of the toric ideal is much more difficult than the kernel of matrix Aj\- 
However we will define the invariance property of 1^^ in terms of the invariance property 
of the kernel of A^- 

As we discuss in Section l?]2] we are interested in the kernel of A^ and the linear space 
spanned by the rows of A a- In the following we denote the kernel of A a and the linear 
space spanned by the rows of Aj\ by ker^A and r{AA), respectively. Note that ker^A 
and t^Aa) are orthogonal complements to each other: t^Aa) = (kery^A)^. 

In statistical theory, r(AA) corresponds to a log-linear model of cell probabilities, 
where the canonical parameter vector of the exponential family is specified to lie in the 
linear space r{Aj\)- We use the single term "model" for A, r[AA) and ker^A because 
they correspond to each other. 

The explicit form of ker^A and ^(Aa) are well known in the literature on contingency 



tables (e.g. 



111). The set ker A^ is written as 

ker AA = {y\ y^iio) = 0, Vin e Xd, G facet(A)}. (1) 



For D C [m], let : T^d Q denote a function defined on the set of marginal cells 1o- 
Then extend the domain of to I by 9£){i) = Ooiio)- We call 6d a function (or a table) 
depending only on the marginal cell io- Let Ld = {^z?} C denote the linear space of 
these tables. Then 

Defacet(A) 

where the summation on the right-hand side denotes the subspace spanned by {L^} DGfacct(A)- 
Note that if G A, then Le C Ld for some D G facet (A). Therefore the right-hand is 
spanned hj Le, E E A. 

2.2 The group of invariance of a toric ideal 

Now we give a definition of the group of invariance of a toric ideal. 

Let 5*1 denote the symmetric group onX, i.e. an element g E Sj is a. permutation of the 
cells of I. Then g E Sj acts (from the left) on the |X|-dimensional rational vector space 
Q'"^' = {(y(*))iex} by the permutation of components: {gy){i) = y{g~^{i)). Similarly g 
acts on the set of indeterminates {ui}i^x. If we regard g as a linear map from Q'-^' to itself, 
then it is represented by a permutation matrix. We denote the permutation matrix also by 
g. Note that g is orthogonal. For a given subspace L C Q'"^', let Gl = {g ^ S\x\ \ gL = L} 
denote the set-wise stabilizer of L. 

Let A he a u X p rational matrix as in the previous subsection. The symmetric group 

2| we defined the group of invariance 
for A as the set-wise stabilizer GkerA C Sp of kei A. From the viewpoint of toric ideal, 
the group of invariance is the set of permutations of the indeterminates, which leaves the 
toric ideal invariant. Let r{A) C Q'' denote the linear space spanned by the rows of A. 
By Proposition 1 of [2|], we have GkerA = Gr{A)- 

Our objective is to understand Gker^A = ^r(yiA) of ^ hierarchical model A. 



Sp acts on the set of columns of A and on Q^. In 
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2.3 A perturbation lemma 

Here we present the following lemma. 

Lemma 1. (Perturbation lemma) Let n,b be positive integers. There exist n positive 
integers (yi)JLiJ such that 

n 

{-b, -6 + 1, . . . , 6 - 1, 6}" 9 (q)[Li ^ Yl 

1=1 

is infective. Furthermore we can choose n vectors Y'^^^ = (Y'/-''')[L-^, j = 1, . . . ,n, such that 
Ij^ is infective for each j and they constitute a basis of the vector space Q". 

Proof. Let Y^^''^ = (26 + j)'~^, G [n]). By the uniqueness of the base 2b + j expression 
of positive integers, the map (q)/Li ^ Ym=i^i^i^''^ is injective. Furthermore Yi^^\ j = 
1, . . . ,n, are linearly independent in view of the van der Monde determinant. □ 

In view of the above lemma, we define a generic contingency table belonging to r{A/\) 
for a given set of cells X and a hierarchical model A with facet(A) = {Di, . . . , Dk}- 

Definition 1. For n = v = Xl^i \-^d^\ (md b = p = \I\ choose such that ([^ 

is injective. Decompose (ll)JLi ^'^^^ subvectors of sizes ITdJ; k = 1, . . . , K , as (5^;)JLi = 
((^Di(*Di))iBieXo,, • • • , {GDK{iDK))iDj,eioJ ^e/ine 

x{i) = Oo.iiDi) H K OdA'^Dk)^ iel. 

We call this x a generic element of r{A/\). 

Note that an element g of the group of invariance Gr(yiA) map a generic element 

X of r{A/\) into r{A/s). This fact helps us to determine Gr{AA)- 

3 Group of invariance of hierarchical models 

In this section we first consider a candidate group for the group of invariance Gker^A ^'^^ 
then present our main theorem, which states that the candidate group is indeed the group 
of invariance, provided that the number of levels Jj, j G [m], are generic. 
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For D C [m] consider a simplicial complex A^, which consists of all subsets of -D, and 
let ker^AD = = {y \ y'^iio) = Oyio G Id}- Then ker^A = flDgfacctCA) ^er by 
([1]). Let Gd = GkcrA^^, denote the group of invariance for A"^. Then it is easily seen that 

fl GdC GkerA^. (4) 

Defacct(A) 

Therefore we can take nDefacct(A) '-^-D as a candidate group for the group of invariance 
G^kcrAA- we will present an example of sudoku in Section [6l in general the inclusion in 
(jlj) is strict. However if the number of levels Ij, j G [m], are generic, then the inclusion 
in (jl]) is in fact an equality. 

Before stating our main theorem, we prove that Gd = Gkor is a wreath product of 
symmetric groups. Let Sxj^ denote the symmetric group acting on the set of D-marginal 
cells and Sj^^ denote the symmetric group acting on the set of Z)'-^-marginal cells, where 

is the complement of D. Let {Sj^cY^ denote the set of all functions iiomlo to Sj^^. 
Then the wreath product Sj ^ wr is a set W = Sx^x {Sj cY°. The operation of W 
as a subgroup of Sj is defined by its action to X, where g = {h,h) (^W acts on i G X by 
{gi)D = hio and {gi)DC = h{i£))iDC. Then we have the following proposition. 

Proposition 1. The group of invariance Gd for the hierarchical model A^ is given by 
the wreath product Sj^^ wr Sj^ . 

Proof. For notational simplicity, we prove the proposition for the case of m = 2 and 
D = {1} and write i as We denote and Sj^^ by Sj-^ and Sj^, respectively. The 

proof for a general case is totally the same by the consideration of a "pseudofactor" (see 
Section m for details on pseudofactors) . 

First we show that Sj^mSj-^^ C Gd. Let x G r{A^D) = L^- Then x{i,j) = 6{i) for 
some 9. Let g G S'/jWrS'/^. Then g{i,j) = {h{i) , hi{j)) , where h G Sj^ and hi G Sj^ for 
each i G [Ji]. Then 

{gx){t,j) = x{g-\z,j)) = e{g-\z,j)^) = e{h'\^l 
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where the subscript "1" in g~^{i,j)i denotes the first component. Therefore gx G Lo- 

We now show the converse Gd C Si^wiSi-,^. In order to show this we assume that 
X G Ld is generic, i.e. 9{iys are distinct. Suppose that {gx){i,j) = 0{g^^{i, G Ld- 
Then g~^{i,j)i does not depend on j. Therefore we can write g~^{i,j) = {h{i),h{i,j)). 
Since g is a. bijection, /i is a bijection and j t— > h{i,j) is a bijection for each i. Therefore 
g'^ e Sj^wi Si,. □ 

Now we state the main theorem of this paper. 

Theorem 1. Consider a hierarchical model A. Assume that \Id\, D G facet(A), are 
distinct and Ij > 2 except for at most one j G [m]. Then the group of invariance GkcrAA 
is given by 

G],eTA^= n {Si^^WlSi^). (5) 
Z)efacct(A) 

A proof of this theorem is given in Section [5] after we estabhsh several important 
facts on hierarchical models in Section HI As seen from the statement of Theorem [H it 
seems that the case of two-level factors Ij = 2 needs a special consideration, although the 
requirements on the levels in Theorem [1] may be too restrictive. We discuss these points 
again in Section [71 We will give some examples of Theorem [1] in Section [6] after rewriting 
the right-hand side of ([5]). 

4 Some basic facts on hierarchical models 

In this section we establish basic facts on hierarchical models. In particular we are in- 
terested in the behavior of a hierarchical model, when a maximal simplex is deleted from 
facet(A). This is because for our proof of Theorem [1] we employ the induction on the 
number K = \ facet (A) | of maximal interaction effects in A. 
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Let E C [m]. We first define "incremental subspaces" of by 

(6) 



if 7^ 0, and = Lq. Recall that Le is the linear space of tables depending only 
on the marginal cell and that J2j&E is the subspace spanned by {LE\{j}}jeE- 

The following lemma is easily proved and well known in statistical analysis of variance 
(ANOVA). 

Lemma 2. Let E and F he subsets of [m]. Then 

(1) NE = LEn{Y^^^j,L^^^\{,})^. 

(2) IfE^F, then Ne^Nf- 

(3) Le = Y.fceNf. 

(4) For any simplicial complex A, r^A^) = ^f&a '^"'^ ker = XIf^a 

(5) The orthogonal projection ttjve onto Ne is given by 

FcE ' 

for all X G Q"^. Recall that \E \ F\ is the cardinality of E\F. 

Let D G facet (A) be a maximal simplex. As in the beginning of Section [3] let 
denote the simplicial complex consisting of all subsets of D. Note that r{Aj^D) = Lo- 
Now we define A\e) by "deleting the maximal interaction effects D from facet(A)", i.e. by 

facet (A\b) = (facet (A)) \ D. 

We have the following proposition. 
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Proposition 2. Let D G facet(A). Then 

r(v4A) nker^A^^ = r(AAD) nker^A^^ = ^ iVg. (7) 

Proof. By Lemma [2|, we have r(y4A) = XleeA and ker Aa^^^ = Xls^A^^ Therefore 
the equahties follow from the relation A \ A\d = \ A\o- □ 

We next define a partial difference operator. For j G [m] and x = {x{i))i(zi define 

{djx){i) = x{i) - x{ii, . . ■ ■ .,im), i = {h, ■ ■ - .im)- 

For E C [m] define Oe = Wj^E^i- Note that for two subsets D,E C [m], E D, we 
have 

dEOD = 0, yen e Ld. (8) 

It is obvious that for any D,E G [m] and 6^) € L^, we have OeOd ^ Le,. 

Concerning the partial difference operator Oe we have the following proposition. 

Proposition 3. For all E C [m], kerdE = '^e^e^f- 

Proof. We first show that the subspace kerc^j is equal to L[m]\{j}. Let x G kerc^j. Then 
x{i) = x{ii, . . . , 1, ij+i, . . . , im) and therefore x G L[.m]\{j}. Conversely, if x G L[m]\{j}, 
then djX = 0. Therefore we see that kerc^j = L[m]\{j}- Since the operators {dj}j^[m] are 
mutually commutable projectors (and therefore simultaneously diagonalizable) , we have 
kerdE = J2jeE^^^ ^j- Therefore, by using Lemma [21 

ker dE = Y^ = Yl ^M\{i} = Y1 Np = ^i"' 

jeE jdE jgE^cHUi} F^E 

The last equality comes from the fact that F ^ E ii and only if F C [m] \ {j} for some 

jeE. □ 

Combining Lemma [2] and Proposition [3l we have the following proposition. We will 
use the proposition with A' = A\e in the proof of the main theorem. 
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Proposition 4. Let A and A' be two simplicial complexes such that A D A' . Then 
X G r{Aj\') if and only if x & '"(^a) and OeX = for all E E A \ A' . 

Proof. The statement is equivalent to r{A^') = r{A^) fl (nsgA\A' ker c^e). Tlie left-liand 
side is J^peA' The right-hand side is 

\F<^A / eA\A' F^E J FeA" 

where A" = {F e A \ F ^ E, \/E e A \ A'}. It is sufficient to prove that A' = A". 
Let F G A'. Clearly F G A. Now assume that there exists some £" G A \ A' such that 
F D E. Then, since F G A' and F D E,we have E G A'. This contradicts to F G A \ A'. 
Therefore F ^ E for any F G A \ A'. Conversely, suppose that F G A and F ^ E for 
any F G A \ A'. Then, since F G A and F ^ A \ A', we have F G A \ ( A \ A') = A'. □ 

In the proof of Proposition [H we treated the combination of factors in D as a single 
factor and the combination of factors in as another single factor. This identification 
is well known in design of experiments as a "pseudofactor" (e.g. 
of this section we fully discuss the notion of a pseudofactor and a natural partial order 
induced on the set of pseudofactors from the hierarchical model. The resulting poset plays 
an essential role in the next section. 

For each i G [m] let st/\{i) = {D G A | {i} U D E Aj denote the closed star of {i}, or 



121]). As the last topic 



equivalently the cone over the link of vertex i (see e.g. 10, Definition 2.14]). Let 



fstA(z) = facet(stA(0) = {D e facet(A) \ i e D} (9) 

denote the facets of st^ii)- If fstA(^) = fstA(j) we say that i,j belong to the same 
pseudofactor and denote this as i ~ j. The relation ~ is an equivalence relation and [m] is 
partitioned into disjoint equivalence classes. We call each equivalence class a pseudofactor. 
In the framework of this paper, we can replace a pseudofactor by a single factor, although 
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we do not do this in this paper. Let V denote the set of pseudofactors, i.e. V = [m]/ ~. 
For p G P let 

fstA(p) = fstA(O) ^ P- 

Now we introduce a partial order onto V by 

p> p' ^ fstA(p) 3 fstA(p') ^ stA(p) D stA(p')- 

With this partial order V becomes a partially ordered set (poset). We call this poset the 
"pseudofactor poset" induced by the simplicial complex A. 

The pseudofactor poset induced by A is related to the intersection poset. The intersec- 
tion poset Q of facet(A) is the set of intersections of facet(A), that is, Q = {HoesD \ S C 
facet(A)}. The order of Q is the reverse inclusion order: CiDesD < HoeS'D if S C S'. We 
assume [m] G Q just for convenience. We show that there is an injective homomorphism 
from V into Q. In fact, the following lemma holds. 

Lemma 3. Let V{p) = Up'>pp' . Then V{p) = noeistAip)^ ■ Furthermore V is an injective 
homomorphism from V into Q. 

Proof. Let i G V{p). Then there exists some p' > p such that i & p' . This means 
fstA(0 = fstA(p') 3 fstA(p)- Therefore i G \^D&stA{p)F>- The converse is similarly proved. 
Next we prove that V is homomorphic and injective. If p' > p then fstA(p') 3 fstA(p) 
and therefore V{p') = n/)gfst^(p')-D > nDgfst^(p)-D = V{p) from the definition of the order 
of Q. If p' ^ p, then Vip') = Up.>p.p" ^ Up.>pp" = V(p). □ 

We remark that V is not surjective in general. For example, let m = 3 and facet (A) = 
{{1, 2}, {2, 3}, {1, 3}}. Then V = {{!}, {2}, {3}} with a trivial order (i.e. no two distinct 
elements are comparable) and Q = {0, {!}, {2}, {3}, {1, 2}, {2, 3}, {1, 3}, {1, 2, 3}}. The 
homomorphism is — {0 ^r z G {1,2, 3}. Thus V is not surjective. In other words, 

the poset Q has the same amount of information as facet (A) because facet (A) = facet (Q\ 
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{[m]}), but the poset V loses the information as the example shows. For description of 
the group of invariance, we only need the pseudofactor poset rather than the intersection 
poset. 

5 A proof of the main theorem 

Now we employ induction on i^' = | facet (A) |. The theorem is true for K = Ihy Propo- 
sition [1] Therefore assume that the theorem holds for K — 1. Throughout the proof we 
choose D e facet (A) such that |Xd| = mini?gfacct{A) \1f\- We consider deleting D from 
facet (A). 

Let X = '^Feiacct(A)^F ^ generic element of r{AA) (Definition [1]). List the values 
of Of as = Opiip) = Opii), ip ^ '^f- Then x{i) can be written as 



X{1} = 

Fefacct(A) jf&If 

In view of (jl]) it suffices to show that any g G Gr{Ai^) belongs to the right-hand side of 



([5]). Fix an arbitrary g G GkcrAA = ^r(AA) ^^^d let y = gx. Then y G r{A/^) and y can 
be written as y = X]Fefacct{A) Vf- Note that at this point we do not have any relation 
between 6'^'s and rjp^s. Fix an arbitrary E' G A \ A\£) and take the partial difference with 
respect to E. Then 

dsvii) = dEVoii) (11) 
by dH]). The right-hand side dETjoii) depends only on i/j. The left-hand side {dEy){i) 
is a linear combination of 21-^1 Z/(j)'s with the coefficient 1 for 2l^l~^ terms and —1 for 
other 21^1"-'^ terms. Now y{j) = x{g^^{j)) = (x o g~^)(^j). We substitute x{g^^{j)) by the 
right-hand side of (fTO!) and take the linear combination. Then {dEy){i) is written as 

Fefacct(A) jF&Ip 

where 

QjAi) = {dEixjF o 9~')m e {-2i^i-\ . . . , 2i^i-i}. 
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Since we have taken generic a^^'s, by the perturbation lemma, Qjp{i) is uniquely deter- 
mined by {dEy){i) for each i and for each F G facet (A) and jp- However recall by ( ITTil 
that {dEy){i) only depends on This implies that Qjpii) also depends only on ic for 
each jp- More precisely, if we take the Id marginal of (|T^ . then we have 

{dEyY{iD) = \lDc\{dEy){i)= ^ <5j,(*i?)aj>- 

Fefacet(A) Jf&^f 

Therefore by uniqueness we see that Qjpii) = (5j^(*_d)/|2^d'^I depends only on i/j. 

Now we claim that Qjp{i) = for all jp, F ^ D, and for all i El. For readability, 
we state this as a lemma and give a proof. Recall that G A \ /S.\o is arbitrarily fixed 
and the following lemma holds for any such E. 

Lemma 4. Qjp{i) = for all jp G Tp, F G (facet(A)) \ {D}, and for all i EX. 

Proof. Suppose that there exists some i° and some jp, such that Qjpii^) 7^ 0. Then, 
because Qjp{i^) only depends on i^)? foi' this jp we have 

\{i I g,,(i) ^ o}| > |{i I iu = = \Xdc\ = 

\-Ld\ 

Write 

= {* I = *d} = {(*DC,«?))}i^cex^c- 
The D'-^-component i^c of the elements of X||0^ are all distinct. 

For i E 1\jQ^, consider Qjp{ip)C, i^j) = (dpixjp ° g~^)){'i'DC, which is a sum of 21^' 
terms of the form ±Xj> ((7"^ (*'))• Since the operator Oe only touches indices ij,j E E G D, 
we note that these terms have the common index ip,c, i.e., i'j^c = *dc^- 

Therefore we can write 

QjpiiDC,iD)= J2 f^3DXjFi9~\iDC,jD)), (13) 

where G {—1,0, 1}. It is important to note that the sets of cells {g~^{iDC,jD)}jDeiD 
are mutually disjoint for different values of Idc, because is a bijection on X. 
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Now if Qjp{iDC,i%) 7^ 0, there exists at least one non-zero term on the right-hand 
side of ( IT3l) . Therefore for each i^c there exists jo such that XjpiQ^^i'^DC , Jd)) = 1- By 
the disjointness noted above, it follows that 

On the other hand, by the definition of Xjpi have 

I xUa-'ii')) = 1}| = I Xi.ii') = 1}| = \1fc\. 

Combining the above results we have 

|X| Ijl 

\Tpc \ > |X£)c| = or |Xp| < |Xd| 



\^f\ \1d\ 
However we have assumed that |Xd| is the (unique) minimum among \Tf\, F G facet (A). 
Therefore F = D. □ 

From the above lemma, we have 

{dEy){i)= Yl Qjnii)^io- (14) 

We have shown fll4p for generic x. However, since (1141) is an algebraic relation and all 
generic tables span r^A^) by the perturbation lemma, (fT4|) holds for all x G r{A/\). Now 
in flTU]) set ttj^ = 0, Wjo G Xd- Namely let x = f &a.cct{A) f^d^f ^'^y element of 
'^(^Ayo)- Then (9^?/ = Oe^x o (7^^) = for all G A \ /S.\d- Therefore y G r(74A^^^) by 
Proposition m This means that g G Gr{A/^) has to map every x G ^(Aa^^o) into r(y4A^^^). 
In other words, g G ^^(Aa^^)- By induction assumption we have shown 

FGfacet(A),F^D 

Now it remains to show that g G Sx^c "^2:13 • By assumption g maps r(y4A) into itself. 
We have shown that g maps r{A/^^^) into itself. Since g is orthogonal as a linear map, it 
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follows that g maps the subspace M = r{A^) fl r^A/^^j^)^ into itself. By Proposition [2l 
we obtain 

M = r{AA)nr{AA^J^ = r{AAo)nkeTAA^^= Yl 

Recall that A'^; is the incremental subspace defined by Note that G M G r(y4^D). 
We claim that there exists a table (pD in M such that (pDiio), *d ^ ^Dj are all distinct. 
We state this lemma and give a proof. 

Lemma 5. There exists a table (pD in M such that (pDiio), *d G ^d, M distinct. 

Proof. Consider a generic element Od of L^. Let tttv^, denote the orthogonal projection 
to No and put (pu = niy^Of). By Lemma [21 the following expression for 0_d(^d) holds. 

Recall that 6^{iE) = J2jeijE=iE ^dUd)- Multiplying each side by |X|, we have 

EcD 

= C{iD,jD)OD{jD), 

3d<^^d 

where 

ECcq{io,jD) 

j£cq{io,jD) 

and eq^iojo) = {j e D \ ij = jj}. Note that CiioJo) e ■ ■ ■ A^\}- For given 

io and i'^, if C^ioJo) 7^ Cii'D^o) for some js, then (poiin) 7^ 0d(«d) because 9d 
is generic. Therefore it is sufficient to prove that if io ^ i'^, then there exists some 
jn G Id such that C{ir),jD) 7^ Since Jj is greater than 2 except for at most 

one i G [m], we can show that C{i£),jD) = \Idc\ Yljeoi^j " ^) '^^^y = Jd- 

Thus C{iD,iD) 7^ C{i'j^,iD) whenever 7^ This proves the lemma. □ 
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We have proved that there exists (po G Nr, C M such that 4>D{iD), in G X/?, are 
all distinct. Since g(f)D G M C r^A/^o), the same proof as in Proposition [1] shows that 
9 e Si^^ wrS'j^. 

This completes the proof of Theorem [H 

6 The wreath product indexed by the pseudofactor 
poset 

Although ([5]) gives a form of the group of invariance, it is not yet sufficiently explicit to 
write down the group of invariance for a given hierarchical model. We can employ the 
notion of a wreath product of a partially ordered set of actions to describe the group 
of invariance more explicitly. The notion of a wrea th p roduct of a partially ordered set 



of actions has been defined by many authors 



definition in Section 7 of 



17|. 



9| 



17| 



15| 



We follow a succinct 



The poset we use is the pseudofactor poset ("P, <) defined in Section HI Recall that V 
is a partition of [m] and each class p E V has fstA(p) = fstA(0 = {D G facet(A) | i G D}, 
for i E p. The order relation p < p' on V is defined by fstA(p) C fstA(p')- Recall that 
V{p) = Upi>pp' . We also define the ancestor set of p by 

A{p) = yjp,^pp' = v{p)\p. 

If A{p) = 0, then we let Ta(p) be a 1-element set, say {1}. 

Definition 2 ( 17|). The wreath product of the symmetric groups {Sxp)pep indexed by the 



poset V is defined by W = ^p6•p('S'I"p)'^■'^*''^ where {Sjp)'^-'^'-p'> is the set of all functions from 
Ta{p) to Sjp. The action of w = {wp)p^v E W on X is defined by 

{wi)p = Wp{iA[p))ip- 

In the above definition, we use the parentheses for evaluating functions (such as 
'*A(p))) and do not use them for action (such as wi). 
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For example, if facet (A) = {D} and C D C [m], then V = {D, D*^} with the order 
relation D > . In this case, the wreath product of {Sxp)pev is the usual wreath product 
Sj^c ^'^D because Ta(d) =A^} ^'^d X4(oC) = Xd. 

The following lemma by J] is useful. 

Lemma 6 (Theorem B of The wreath product is characterized as follows. 

v{p) depends only on iy^p) for any peV] . 

pev 

The proof of the following lemma is easy and omitted. 

Lemma 7. Let A and B he two subsets of [m]. Let g G Sj. Assume that {gi)A depends 
only on and that {gi)B depends only on is- Then {gi)AnB depends only on tAnB, and 
{gi)AuB depends only on iaub- 

Now we establish the following theorem. 
Theorem 2. The group of invariance coincides with the wreath product of{Sxp)pe_v! that 



IS, 



fl (5x,cwr5xj = WiSxf-^^^. (15) 

Defacct{A) peV 

Proof. By Lemma [HI the left-hand side in fllSp is equal to 

{g E Sj\ {gi)D depends only on for any D G facet (A)}. 
On the other hand, also by Lemma [6l the right-hand side in f|T5|) is equal to 

{g ^ Sj \ {gi)Y(^p) depends only on iv{p) for any p eV}. 
Now the equality (fT5|) is clear if one uses Lemma [7] with two relations 

D = U Vip) and Vip) = f] D. 

peV,pcD DefstA(p) 

The former one is from the construction of V. The latter one is Lemma O □ 
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Corollary 1. The group of invariance is equal to the direct product of the symmetric 
groups {Sxp)pev if and only if the posetV has the trivial order, i.e. no two distinct elements 
ofV are comparable. 

Let us present some examples. Here we abbreviate (S'jp)'^'^(''' to "S**!^^^^, and to S"*, 
respectively. 

Example 1. Let facet(A) = {{1}, . . . , {^72}}. Then V = {{1}, • • • , {^72}} with the trivial 
order. The wreath product is the direct product W = YYjLi ^{j}- 

Example 2. Let m = 3 and facet(A) = {{1}, {2, 3}}. In this case, {2, 3} is a pseudof ac- 
tor hut not a single factor. Then the pseudofactor poset is {{1},{2,3}} with the trivial 
order. The wreath product isW = 8*^^^^ x 5*123}- 

Example 3. Let m = 3 and facet(A) = {{1, 2}, {2, 3}}. Then the pseudofactor poset is 
{{1}, {2}, {3}} with the order relations {1} < {2} and {3} < {2} (no other relations). 
The wreath product isW = 5'|]^}|{2} ^ ^^{2} ^ '^{3}|{2}- 

Example 4. Let m = 3 and facet(A) = {{1},{2}}. Note that the factor {3} does not 
appear explicitly. Then the pseudofactor poset is {{1}, {2}, {3}} with the order relations 
{3} < {1} and {3} < {2}. The wreath product isW = S*i^-^^ x 5*12} x 5*13111^2}- 

Example 5. Let m > 3 and facet(A) = {{1, 2}, {2, 3}, . . . , {m — 1, m}, {m, 1}}. Then 
V = {{1}, . . . , {m}} with the trivial order. The wreath product is W = Y[J=i ^{j}- 

Example 6. We give an example in that the number of orbits in the minimal Markov basis 
is smaller under the action of the group of invariance than under the action of the direct- 
product group. Let m = 5 and facet(A) = {{1,3}, {2,4}, {3,4,5}}. This model is decom- 



posable (see \l Aj for the definition) and the minimal vertex separators of the corresponding 
chordal graph are {3} and {5}. The pseudofactor poset is V = {{1}, {2}, {3}, {4}, {5}} 
with the order relations {1} < {3}, {2} < {4}, {5} < {3}, {5} < {4} (no other relations) 
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and the wreath product is given asW = S^^y^^^y x 5'|2|||4| x S^^r^y x S^^y x •S'lgmg ^y . Consider 
the following two moves 

Ml = (11111)(12211) - (12111)(11211), 
M2 = (11112)(12211) - (12112)(11211), 

where the notation follows one in The moves Mi and M2 are indispensable because 
they connect the following two-element fibers, respectively. 

Ti = {(11111)(12211),(12111)(11211)}, 
= {(11112)(12211), (12112)(11211)}. 

The direct-product group 11^=1 ^ij co^nnot map Mi to M2 because the number of the distinct 
levels of the 5th factor in Mi is different from that in M2. On the other hand, the group 
of invariance maps Mi to M2 with a permutation of the 5th factor when the level of the 
3rd factor is 1. 

Note that the Sy^y^y^^^y involves the set {3,4}, which is not a maximal clique nor a 
minimal vertex separator of the chordal graph. This shows that the group of invariance 
can not be described by usual notions of decomposition of a chordal graph. 

Example 7. Let m = 6 anc? facet (A) = {{1, 4, 5}, {2, 5, 6}, {3, 4, 6}}. Then the pseud- 
ofactor poset is V = {{1}, {2}, {3}, {4}, {5}, {6}} with the order relations {1} < {5}, 
{1} < {4}, {2} < {5}, {2} < {6}, {3} < {4} and {3} < {6} (no other relations). The 
wreath product is W = 'S'|i}|{4,5} x 'S'{2}|{5,6} ^ '^{3}|{4,6} ^ "^{4} ^ ^{5} ^ "^{e}- 

The last example is a counter-example to the conjecture in the discussion of {2!, Section 
5]. In our terminology, the conjecture is stated as "If all pseudofactors are single, i.e. V = 
{{!},..., {tti}}, and the intersection of facet (A) is empty, then the group of invariance is 
the direct product of the symmetric groups on each factor". The conjecture is justified if 
we impose an additional condition that V has the trivial order (see Corollary [T]) . 
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We show an example in that the inclusion (jl]) is strict. 

Example 8 (Sudoku). The solution of sudoku a 9 x 9 table whose each row, each 
column and each 3 x 3 block contains the digits from 1 to 9 exactly once. Following the 
terminology of ^ 



'iJ/J . we call a "row" of 3 blocks a band and a "column" of 3 blocks a stack. 
The solution is considered as a3x3x3x3x9 contingency table x{i,j, k, I, c) where we 
define x{i,j, k, l,c) = 1 if the number c G [9] is put on the j-th row of the i-th band and 
the l-th column of the k-th stack and x{i,j, k, l,c) =0 otherwise. Then the restriction is 
given by four equations 

x{i,j,+,+,c) = l, x{+,+,k,l,c) = 1, a;(z, +,/c, +,c) = 1, x{i, j, k,l, +) = 1, 

where "+" denotes taking marginal (sum) over the index. The maximal simplices of this 
model is given by 

facet(A) = {{1,2,5}, {3,4,5},{1,3,5},{1,2,3,4}}. 

The pseudofactor poset is V = {{1}, {2}, {3}, {4}, {5}} with the order {1} > {2} and 
{3} > {4} (no other relations). The wreath product is given by 

W = 5*1^1 X 5'|2}||i} X 5*131 X 5*1411131 ^ 

which consists of permutation of bands, permutation of rows in each band, permutation 
of stacks, permutation of columns in each stack and permutation of numbers. However, 
the group of invariance Gkcr additional permutation f defined by f{i, j, k, I, c) = 

{k,l,i,j,c). The permutation f does not belong to the wreath product W . Note that 
the model does not satisfy the assumption of TheoremU\ because |X{i2,5}| = |X{3_4^5}| = 
|2^{i,3,5}| = |^{i.2,3,4}| = 81- The group generated by W and f is used to count the number 



of essentially different solutions of sudoku in 
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7 Discussions 



We derived an explicit formula of the group of invariance provided that the number of 
levels Ij, j G [m], are generic. In our future work we intend to generalize this result 
by weakening the restriction on the number of levels. We conjecture that under mild 
regularity conditions the group of invariance is generated by the wreath product of this 
paper and the permutation of factors with a common number of levels. However, it seems 
to be difficult to solve this problem. For example, as described in Example 3 of ^, 
the group of invariance for the 2x2x2 contingency tables with fixed two-dimensional 
marginals is different from the new conjectured candidate group. In the example, as was 
pointed out by a referee to ^, the group of invariance is not faithful. Here an action 
G to L is called faithful if the kernel {g E G \ gx = x, Vx G L} of the action consists 
only of the unit element. On the other hand, we can prove that the group of invariance 
is faithful under the assumption of Theorem [H Indeed, in a similar way to the proof of 
Lemma [5], we can show that there exists a table G N[m] C ker^A such that {0(i)}igj 
are all distinct. Therefore if g(j) = (p, then g has to be the identity map. 

Random sampling from the group of invariance is important for performing the Markov 
Chain Monte Carlo (MCMC) method on contingency tables. See 3|] for details. In 
Theorem [2] we rewrote the group of invariance from an intersection form to a wreath- 
product form. The wreath product is useful for random sampling. Let us briefly describe 
it. The wreath product is given hj W = Ylpev^^^p^'^^'''''' ■ show an algorithm to obtain 
a uniformly random sample w = {wp)p^'p from W . Let us number V as V = {pi, . . . , pi} 
such that i < j whenever pi < pj. Then, from i = I down to 1, we independently generate 

'^P^i^A{p^)) fro"^ ^ip^ ^^ch ^ ^A{p^)■ ^hc resultlug element w = {wp)p^-p is a 

uniformly random sample from W. Remark that the intersection form in Theorem [1] does 
not give such a procedure. 
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